Store API, orchestration fields, and per-state concurrency#1
Merged
Conversation
Adds a namespace-based KV store (store table) with full CRUD via REST
endpoints (PUT/GET/DELETE /store/{namespace}/{key}), plus per-state
concurrency limits on lease claims.
- Apply migration 002 (run_id, workflow_state_json columns; drop gate_results)
in store.zig migrate() which was previously skipped
- Add run_id and workflow_state_json columns to 001_init.sql for fresh databases
- Remove gate_results table from 001_init.sql (no longer used)
- Remove dead store functions: listTasks, listEvents, listArtifacts
(replaced by paginated variants)
- Remove all gate_results references from openapi.json, README, docs
- Add store (KV) endpoints and /tasks/{id}/run-state to openapi.json
- Add store schemas (StorePutRequest, StoreEntry) to openapi.json
- Add workflow_id field to PipelineStateDef in openapi.json
- Update README API surface table with store endpoints
- Update docs/api.md, architecture.md, workflows.md to remove gates,
add store section
- Update AGENTS.md and README project layout with missing modules
(types.zig, config.zig, migration files)
INSERT OR REPLACE deleted and re-inserted the row, resetting created_at_ms on every update. Switch to INSERT ... ON CONFLICT DO UPDATE so only value_json and updated_at_ms change. Add test assertion to verify created_at_ms is preserved across upserts.
The response structs in types.zig are unused — api.zig builds JSON responses via manual writer calls instead of serializing these types.
The orchestration migration (run_id, workflow_state_json columns and gate_results drop) is inlined in store.zig's migrate() function. The .sql file on disk was never loaded via @embedfile and served no purpose.
The claim handler accepts an optional concurrency object with a per_state map for per-state concurrency limits, but this was missing from the OpenAPI spec.
Remove types.zig entry and add clarification to store.zig description.
…etry tests Cover the core claim-transition workflow, per-state concurrency limits blocking excess claims, role-based claim filtering, and fail-with-retry policy behavior. All tests use :memory: SQLite and std.testing.allocator.
handleGetTaskRunState freed the store-allocated run_id string using the request arena allocator (whose free is a no-op), causing the GPA allocation to leak. Use ctx.store.freeOwnedString instead.
AGENTS.md referenced src/types.zig which was removed. Added export_manifest.zig and from_json.zig to both AGENTS.md and README.md project layout sections.
gate_results table was dropped in the orchestration migration. The e2e
test still referenced POST/GET /runs/{id}/gates endpoints and expected
transition to fail when required_gates were unsatisfied. These endpoints
no longer exist and gates are now informational only.
Added comprehensive Store API e2e tests covering put, get, upsert,
list namespace, FTS5 search, delete key, and delete namespace.
Add -DSQLITE_ENABLE_FTS5 compile flag so migration 004 (Store FTS5 virtual table) works on all platforms.
- Add sanitizeFts5Query() that wraps each search term in double quotes with internal quote escaping, preventing malformed FTS5 syntax from causing 500 errors - Reject empty/whitespace-only queries with 400 instead of passing to SQLite - Reject "search" as a namespace in handleStorePut with 400 error - Document the /store/search route reservation in routing comments
Decode %XX sequences and '+' as space in search query parameter. Fixes E2E test where q=endpoints+methods was searched as literal string with '+' instead of two separate words.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds namespaced key-value store with full-text search, orchestration integration fields, and per-state concurrency control for task claiming.
Store API
GET /store/search?q=...with optional namespace filterOrchestration Integration
run_idcolumn on tasks — links tasks to workflow runsworkflow_state_jsoncolumn — stores workflow state on tasksupdateTaskRunId,updateTaskWorkflowStatePer-State Concurrency
concurrency.per_statelimitscountLeasedTasksInStatechecks current load before claimingCleanup
gate_resultssystem (table, queries, API handlers, types)Infrastructure